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ABSTRACT 


in  Section  A,  A  Logical  Analysis  of  Guessing,  appropriate  test-taking 
strategies  are  derived  for  six  major  test-scoring  procedures.  Three 
commonly  used  definitions  of  guessing  are  Interpreted  as  corresponding 
degree-of-conf idence  distributions.  The  ability  of  the  testing  pro¬ 
cedures  to  separate  these  distributions  from  those  representing  higher 
degrees  of  knowledge  is  considered  with  the  major  result  that  only 
admissible  probability  measurement  performs  satisfactorily. 


In  Section  B,  The  Effect  of  Guessing  on  the  Quality  of  Personnel  and 
Counseling  Decisions,  the  fundamental  probability  distributions  for 
total  test  scores  are  derived  by  assuming  that  each  person  knows  the 
answers  to  some  items  and  guesses  on  the  remaining  items.  Analysis  of 
a  10-item  test  shows  that  guessing  levels  encountered  in  practice 
(a)  seriously  degrade  the  value  of  selection,  placement,  and  counseling 
decisions,  (b)  significantly  impair  test  reliability  and  validity,  and 
(c)  magnify  the  influence  of  testwiseness. 


in  Section  C,  The  Worth  of  individualizing  instruction,  equations  are 
developed  for  expressing  the  cost  and  gain  for  applying  an  instructional 
sequence.  The  expected  return  from  assigning  instruction  on  the  basis 
of  (1)  admissible  probability  measurement,  (2)  admissible  choice  testing, 
(3)  conventional  choice  testing.  (4)  prior  information  only,  and 
(5)  matching  the  average  student  is  computed  for  each  of  seven  distri¬ 
butions  of  state  of  knowledge.  The  performance  of  (I)  is  outstanding; 
that  of  (2),  (3) i  and  (4)  is  disappointing,  while  (5)  does  surprisingly 
well. 


The  Wt>rth  of  individualizing  Ihstructlon 


It  would  be  surprising  If  the  development  of  a  decision-theoretic  psychometrics 
were  not  able  to  provide  some  useful  Insights  Into  the  process  of  Instruction. 
Instruction,  whether  guided  externally  or  Internally  Is  basically  a  cybernetic 
process  and  an  essential  feature  of  this  process  Is  Information  concerning  the 
current  state  of  a  person's  knowledge  (Shuford  6  Massenglll,  1965).  Now,  a  decision- 
theoretic  psychometrics  should  be  concerned,  generally,  with  techniques  for  obtaining 
and  utilizing  information  about  the  current  state  of  a  person's  knowledge.  A  suf¬ 
ficiently  developed  decision-theoretic  psychometrics  could  be  viewed  as  a  theory  of 
instruction.  However,  in  our  judgement,  though  decision  theory  would  provide  an 
excellent  framework  within  which  to  develop  a  theory  of  Instruction,  too  much  Is 
buried  in  the  utility  functions  of  decision  theory,  it  Is  quite  feasible  to  develop 
a  theory  of  utility  functions  for  a  certain  area  of  application  (Toda  S  Shuford,  1965) 
Such  a  theory  serves  to  provide  a  rational  basis  for  deriving  utility  functions  from 
rather  simple  premises  and  greatly  reduces  the  measurement  problems  involved.  A 
theory  of  utility  functions  to  b^  used  in  the  area  of  Instruction  clearly  must  incor¬ 
porate  a  theory  of  learning.  Theories  of  learning  logically  consistent  and  coherent 
with  decision-theoretic  psychometrics  have  been  developed  to  a  considerable  extent 
(Shuford,  196**;  Watanabe,  I960;  1966).  The  real  potential  of  a  decision-theoretic 
theory  of  instructkn  will  be  realized  when  it  has  been  developed  to  the  extent  that 
it  can  deal  efficiently  with  complex  structures  with  the  hierarchical  organization 
of  knowledge  (Toda  6  Shuford,  1965;  Watanabe,  1966).  Unfortunately,  we  are  some 
distance  away  from  achieving  such  a  full-fledged  decision-theoretic  theory  of  Instruc¬ 
tion. 

So,  now  let  us  return  to  a  more  mundane  level  of  where  we  are  now.  In  previous 
work  (Massengill  &  Shuford,  1965),  we  considered  a  very  simple  Instructional  situation 
where  the  teacher  had  to  decide  whether  a  person  was  uninformed  or  well-informed. 

The  formal  explication  just  allowed  two  states  of  knowledge  for  the  person  and  the 
two  instructional  sequences  were  specifically  tailored  for  these  two  states  of 
knowledge.  Comparison  of  admissible  with  conventional  choice  procedures  indicated  a 
superiority  for  the  former  which  was  of  considerable  significance  under  a  wide 
variety  of  conditions. 

The  results  In  Section  fl  of  this  report  concerned  with  placement  decisions  can 
also  be  viewed  as  relevant  to  instruction,  since  the  two  groups  separated  by  the 
placement  process  can  be  given  different  instructional  sequences.  In  contrast  to 
our  earlier  paper,  the  placement  problem  allows  a  greater  number  of  degrees  of 


knowledge  and  for  more  general  utility  functions.  Still,  the  utility  functions 
are  rather  arbitrary  and  empirical  and  we  propose  that  In  this  section  of  the 
report  to  use  more  rationally  derived  utility  functions,  but  with  essentially  a 
one-item  test.  Though  these  results  are  still  some  distance  away  from  the  full- 
fledged  dec  Is  Ion- theoretic  theory  of  Instruction,  they  should  provide  some  Insight 
Into  many  Instructional  situations  and  they  should  provide  the  basis  for  a  more 
thoroughly  developed  theory  of  instruction. 

PRECISELY  TAILORED  INSTRUCTION 

Suppose  that  instruction  can  be  precisely  adjusted  to  a  person's  state  of 
knowledge  as  represented  by  p.  For  example,  If  a  person'  is  thoroughly  misinformed 
I .e.  p»0,  then  the  appropriate  Instructional  sequence  devotes  considerable  effort 
attempting  to  get  the  person  to  unlearn  false  notions  and  to  building  up  the 
person's  confidence  In  the  correct  concept.  If,  on  the  other  hand,  the  person  is 
uninformed,  i.e.,  p  Is  approximately  equal  to  .50,  then  no  unlearning  Is  necessary 
and  the  Instruction  concentrates  on  building  up  the  person's  confidence  In  correct 
concepts.  And  finally,  at  the  other  extreme.  If  a  person  Is  completely  confident 
or  certain  of  the  correct  answer,  i.e.,  p-1 ,  no  instruction  is  required.  Now, 
the  cost  of  these  Instructional  procedures  decreases  as  the  person's  degree  of 
confidence  Increases.  The  cost  of  Instruction  decreases  down  to  the  point  of  no 
cost  If  the  person  is  completely  confident  In  the  correct  answer.  Now,  cost  may 
be  composed  of  many  components,  such  as  the  student's  and  the  teacher's  time  which 
at  least.  Implies  that  time  Is  used  that  might  be  devoted  to  learning  additional 
topics.  There  Is  also  the  cost  of  Instructional  materials.  This  latter  cost  may 
be  considered  as  sunk  cost  and,  therefore,  would  not  enter  Into  the  calculations. 
Thus,  student  and/or  teacher  time  Is  the  primary  component  of  this  cost  function 
and  might  be  approximated  by  assuming  that  it  Is  proportional  to  1  -  p.  If  c  is 
the  maximum  cost  Incurred  when  the  student  Is  thoroughly  misinformed  (p  ■  0), 
then  the  cost  function  may  be  expressed  as  -c(l  -  p)  and  graphically  represented 
as  shown  In  Graph  A  of  Figure  1. 

Consider  now  the  gain  resulting  from  instruction.  Assume  first  that  all 
precisely  tailored  Instructional  sequences  are  completely  effective.  As  a  conse¬ 
quence  of  experiencing  a  precisely  tailored  instructional  sequence,  a  student  with 
an  Initial  state  of  knowledge  represented  by  p  will  terminate  the  sequence  with  a 

state  of  knowledge  represented  by  p  ■  1,  I.e.  he  will  be  certain  of  the  correct 

* 

concept. 
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INDIVIDUAL'S  STATE  OF  KNOWLEDGE 


A  misinformed  person  Is  less  effective  than  an  uninformed  person,  since  a 
misinformed  person  will  go  ahead  and  confidently  take  action  on  the  basis  of  his 
misinformation,  while  an  uninformed  person  will  be  motivated  by  his  uncertainty 
to  seek  more  information  before  taking  action  or,  If  this  is  not  possible,  he 
prefers  to  take  less  extreme  action.  And  r.ore  clearly,  perhaps,  a  well-informed 
person  Is  more  effective  than  an  uninformed  person  (more  effective  In  taking 
actions  which  better  serve  his  own  ends).  Taking  all  this  into  account,  it  Is 
evident  that  the  gain  from  using  a  precisely  tailored  instructional  sequence  is 
greatest  for  misinformed  students  .,<k!  declines  down  to  zero  for  completely  Informed 
students.  For  the  sake  of  simplicity,  assume  that  the  gain  from  the  use  of  a 
precisely  tailored  instructional  sequence  is  proportional  to  1  -  p.  In  other 
words,  this  gain  function  can  be  represented  by  a  straight  line, with  the  gain 
associated  with  changing  a  thoroughly  misinformed  student,  (p  ■  0),into  a  thoroughly 
Informed  student, (p  ■  1),'  twice  that  of  changing  an  uninformed  student, (p  ■  |), 
into  a  thoroughly  informed  student.  [Note:  This  assumption  is  not  always  approp¬ 
riate,  since  there  are  probably  Instances  in  which  being  misinformed  is  many 
times  worse  than  being  uninformed,]  This  gain  function  can  be  represented  graphi¬ 
cally  as  in  Graph  B  of  Figure  I  with  g  equal  to  the  maximum  possible  gain. 

Mow,  by  adding  the  cost  and  gain  functions  in  order  to  subtract  the  cost 
from  the  gain,  a  linear  totaJ  return  or  net  gain  function  is  obtained,  which  is 
proportional  to  1  -  p  with  a  maximum  of  g  •  c  at  p  ■  0  and  with  a  minimum  of  0 
at  p  ■  I.  This  is  represented  graphically  In  Graph  C  of  Figure  1. 

If  a  student's  state  of  knowledge,  p,  is  determined  with  some  precision,  as 
for  example,  by  using  an  admissible  probability  measurement  procedure,  then  it  Is 
possible  to  think  of  precisely  tailoring  instruction  to  his  state  of  knowledge . 
Instruction  can  be  matched  precisely  to  the  needs  of  the  student.  Such  a  procedure 
has  often  been  considered  a  desirable  goal.  In  the  remainder  of  this  section, 
the  hypothetical  desirability  of  this  goal  will  be  computed  and  shown  to  be  a 
function  of  the  student's  states  of  knowledge.  The  actual  quantities  obtained 
are  not  important  In  themselves  and  do  not  represent  the  real  value  of  precisely 
tailored  instruction,  since  utility  values  will  be  assigned  arbitrarily  without 
empirical  determination  or  without  derivation  from  a  theory  of  learning.  What  Is 
Important,  however,  is  that  In  later  sections  the  performance  of  other  procedures 
will  be  compile**!  ond  i ompa rod  to  tb I s  best  possible,  ideal  performance  of  an 
t net i uct lonal  system.  The  relative  values  of  these  perfor"ance  indices  are 


meaningful  and  remain  valid  once  the  actual  utilities  are  empirically  determined. 

The  return  from  the  use  of  precisely  tailored  instructional  sequences  Is  a 
function  of  a  student's  state  of  knowledge  as  represented  by  p.  Therefore,  the 
total  return  realizable  by  processing  a  number  of  students  depends  upon  the  dis¬ 
tribution  of  p  for  this  group  of  students. 

If  there  are  M  students  in  this  group  and  if  the  state  of  knowledge  of  the 
i-th  student  is  represented  by  pj,  then  the  total  return,  R,  from  precisely 
tailored  instruction  is 

N  N 

(1)  R  -  l  (g-c)O-p.)  -  (g-c) (K-  l  p.), 
i«l  1  i«l  1 

For  dealing  with  large  numbers  of  students  and  for  theoretical  purposes,  it 
is  convenient  to  represent  the  distribution  of  the  student's  states  of  knowledge 
by  a  continuous  probability  distribution  defined  over  the  interval  [0,1].  A 
reasonably  flexible  distribution  is  the  Beta  function  which  depends  upon  two  para¬ 
meters,  a  and  b,  which  must  be  greater  than  zero. 


(2)  p(p)  -  yP) 
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The  mean  of  this  distribution  is 


(3)  P  -  E(p)  - 

and  the  variance  is 


(*)  V(p) - - . 

(a+b) 2(a+b+l) 

Some  representative  Heta  functions  are  graphed  in  Figure  2. 

The  expected  return  per  individual,  r,  from  precisely  tailored  instruction  with  ? 
group  of  students  whose  states  of  knowledge  are  represented  by  the  Beta  function  is 

(5)  r  ■  /  (g-c)O-p)  fg(p|a,b)dp 
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Figure  2.  Illustrative  distributions  of  state  of  knowledge  concerning  topic  to  be  taught. 
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■  (g-c)d-p). 

The  expected  return  per  individual,  r,  depends  only  on  the  mean  of  the  distribution 
of  the  students'  states  of  knowledge  and  is  a  decreasing  function  of  this  mean. 

It  does  not  depend  upon  the  variance  of  the  Beta  function.  However,  if  the  return 
function  were  non-linear  then  the  expected  return  per  individual  would  depend 
upon  the  variance  and,  possibly,  upon  other  moments  of  the  distribution  of  the 
students'  states  of  knowledge.  This  is  true  regardless  of  the  exact  mathematical 
form  of  the  distflbdteoo. 

A  PRIORI  CHOICE  BETWEEN  INSTRUCTIONAL  SEQUENCES 

Now,  consider  the  use  of  choice  methods  to  indicate  a  student's  state  of 
knowledge.  In  abandoning  precise  measurementof  a  student's  state  of  knowledge, 
we  must  give  up  all  hope  of  using  precisely  tailored  instructional  sequences,  since 
there  is  no  basis  for  matching  them  to  the  student's  state  of  knowledge.  Ue  must 
choose  from  among  several  instructional  sequences  and  there  is  no  point  in 
choosing  more  than  the  number  of  discriminations  we  can  make  in  a  student's  state 
of  knowledge.  Remember  from  Section  A  above  that  choice  methods  are  generally 
capable  of  discriminating  at  most  three  states  of  knowledge;  informed,  uninformed, 
and  misinformed.  Thus,  conventional  choice  testing  can  only  make  the  discrimination 
not  misinformed  vs.  not  we  1 1-  informed.  However,  we  will  be  considering  admissible 
choice  procedures  in  the  next  section  and  since  admissible  choice  can  distinguish 
three  states  of  knowledge,  we  will  use  three  Instructional  sequences  for  both  the 
conventional  choice  and  the  admissible  choice  procedures. 

As  for  the  instructional  sequences,  we  will  select 

lj  =  the  precisely  tailored  instructional  sequence  for  p»l  (No  instruction). 

I l/2  =  the  precisely  tailored  instructional  sequence  for  p»)/2  (A  moderate 
amount  of  instruction,  no  uniearning). 

Iq  =  The  precisely  tailored  instructional  sequence  for  p»0  (Extensive  unlearn¬ 
ing  of  material,  moderate  instruction  in  the  correct  concepts). 

Now,  we  must  make  some  assumptions  about  what  happens  when  an  instructional 


t 


sequence  is  misapplied  (There  was  no  reason  to  do  this  before  when  we  could  be 
certain  of  precisely  matching  instructional  sequences  with  the  student's  state  of 
knowledge) . 

The  case  of  no  Instruction,  1^,  is  relatively  straightforward,  it  should 
result  in  no  cost  and  no  gain  and  can  be  graphed  as  shown  in  Figure  3.  It  is, 
of  course,  a  horizontal  line,  r  ■  0. 

For  the  case  of  moderate  instruction,  l,/9,  the  cost  will  be  constant  at  a 

1  1  1 
value  of  rc,  while  the  gain  will  decline  from  a  maximum  of  rg  at  p  ■  r  as  p 
L  j  Z  Z 

increases  toward  1.  As  to  what  happens  for  p  <  j,  It  seems  reasonable  to  assume 
that  instruction  would  become  less  effective  as  a  student  becomes  more  convinced 
of  an  incorrect  concept,  if  a  student  were  thoroughly  convinced  in  the  correctness 
of  a  wrong  notion,  it  appears  likely  that  Such  training  would  have  no  effect  at 
all  on  the  student's  state  of  knowledge,  since  any  demonstration  of  the  wrongness 
of  his  idea  would  be  lacking.  Thus,  the  gain  from  applying  '  1  /2  decreases  as  p 
goes  from  1/2  down  to  0  and  the  gain  at  p  ■  0  is  nil.  For  simollcity,  assume 
that  this  decrease  is  linear.  Then,  the  resulting  function  can  be  graphed  as 
shown  in  Figure  3.  it  is  a  triangular-shaped  function  with  one  maximum  at  p  ■  1/2 
and  with  two  minima  at  p  »  0  and  p  *  1.  The  minimum  value  is  -  |c. 

Finally,  consider  the  effect  of  applying  the  instructional  sequence,  1^, 
tailored  for  the  thoroughly  misinformed  student.  The  cost  is  constant  at  c  (This 
is  obviously  an  approximation  since,  in  some  cases,  a  better- informed  student 
can  complete  a  sequence  more  rapidly,  leading  to  a  lower  cost,  but  there  probably 
exists  some  additional  cost  In  lost  motivation  or  boredom  as  a  result  of  suffer¬ 
ing  through  inappropriate  instruction).  Mow,  ar.  p  increases,  the  result  of 
applying  iQ  is  over-instruction,  in  the  sense  that  the  instruction  is  more  than 
sufficient  to  carry  the  student's  state  of  knowledge  from  p  up  to  p  ■  1.  Since 
complete  certainty,  represented  by  p  ■  1 ,  Is  the  limit  on  a  student's  knowledge 
with  respect  to  a  particular  concept  or  topic,  the  gain  from  applying  iQ  must 
decrease  as  the  student's  initial  state  of  knowledge,  p,  moves  toward  p  ■  1. 

Again,  we  assume  for  the  sake  of  simplicity  that  this  decrease  is  linear  in  1  -  p. 
Adding  these  cost  and  gain  functions  we  obtain  the  return  function  for  lg  shown 
in  Figure  3.  Notice  that  it  is  a  linear  function  with  a  negative  slope  and  with 
a  maximum  of  g  -  c  at  p  ■  0  and  a  minimum  of  -c  at  p  ■  1 . 

The  dashed  line  in  Figure  3  is  the  return  function,  1^,  for  precisely 
tailored  Instruction  as  shown  previously  in  Figure  I.  in  applying  one  of  these 
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Figure  3.  Return  functions  from  three  rigid  Instructional  sequences, 
|q,  Ij/2  *  ,  and  f°r  precisely-tailored  Instruction,  lQ. 


three  rigid  sequences,  there  is  no  loss  relative  to  precisely  tailored  Instruction 
at  the  three  values  of  p  ■  0,  p  ■  1/2,  or  p  ■  1.  At  other  values  of  p  there  is 
some  loss  and  the  use  of  these  three  rigid  instructional  sequences  treat  both 
moderately  misinformed  and  moderately  well-informed  students  rather  badly,  although 
we  can  do  no  better  without  having  more  precise  information  (such  as  that  obtained 
through  admissible  probability  measurement)  about  each  student's  state  of  knowledge. 

Sometimes  it  may  happen  that  we  have  precise  information  about  a  student's 
state  of  knowledge,  but  because  of  economic  or  other  reasons  we  may  be  able  to  apply 
just  one  of  the  instructional  sequences,  for  example,  we  may  have  available  three 
programmed  texts  at  different  difficulty  levels  or  three  textbooks,  elementary, 
Intermediate  and  advanced.  In  this  instance,  examination  of  figure  3  clearly 
indicates  that,  for  students  who  are  more  than  moderately  misinformed,  the  best 
assignment  Is  the  instructional  sequence  1^,  whereas  for  students  who  are  more 
than  moderately  well-informed,  the  best  instructional  sequence  Is  1^,  which  usually 
represents  no  instruction,  while,  for  all  other  students,  the  return  would  be 
maximized  by  assigning  them  instructional  sequence  Consider  now  that  we  have 

no  test  information  on  each  student,  but  we  have  the  overall  distribution  of 
ability  levels  as  expressed  by  the  ?eta  function,  l/hich  is  the  best  of  the  three 
instructional  sequences  to  apply?  This  same  sequence  will,  of  course,  be  applied 
to  all  of  the  students.  This  analysis  wl 1 1  serve  to  illustrate  the  logic  of  the 
institutional  decision  involved  and  can  also  provide  a  baseline  against  which  we 
can  compare  the  performance  of  the  choice  testing  procedures  which  will  be  con¬ 
sidered  later.  We  need  to  compute  the  expected  return  from  applying  each  of  the 
three  instructional  sequences.  Consider  not  applying  lj,  that  is,  no  instruction, 
the  action  most  appropriate  If  a  student  is  thoroughly  informed.  Since  the  return 
for  all  values  of  p  of  applying  1^  is  0,  the  expected  return  will  also  be  zero, 
regardless  of  the  shape  of  the  distribution  of  states  of  knowledge, 

(6a)  E  I  j  =  0. 

The  expected  return  from  applying  lg  Is  obtained  by  weighting  the  return  at 
each  level  of  p  by  the  corresponding  probability  density  and  integrating  this 
function  over  the  interval  [0,1].  Thus, 

(6b  E  lg  “  /  [g(l-p)-c]fg(p|a,b)dp  »  g(l-p)  -  c. 


Here  again  the  expected  return  depends  only  upon  the  mean  and  not  upon  any  other 
moment  of  the  distribution  of  the  students'  states  of  knowledge. 

Consider  no w  app.lyinq  instructional  sequence  i^2*  before,  we  w's^  to 
integrate  the  product  of  the  return  function  of  the  probability  density  over  the 
whole  range.  Since  this  is  a  triangular  function  with  one  discontinuity  at  p  ■  1/2, 
we  can  break  the  Integral  into  two  parts  and  perform  the  corresponding  integration 
by  parts 

(6c)  E  i1/2  «  /|  (gp  -  |)fg(p|a,b)dp  +  [g( 1 -p)  -  |]fg(p|atb)dp 

«  g{pGb(a+l  |  ^,a+b)  +  (1-p)  [Gfa(b+1  ,a+b)  1 }  -  -. 

The  second  line  of  this  equation  is  given  for  convenience  in  obtaining  actual 
numerical  values  for  these  integrals.  stands  for  the  right-hand  cumulative 
of  the  binomial  distribution.  The  incomplete  integral  of  the  Beta  distribution 
can  now  be  solved  explicitly  in  terms  of  a  and  b.  Thus,  tables  have  been  computed 
and  published  for  the  cumulative  of  the  Beta  distribution,  however,  it  is  more 
convenient  and  more  accurate  in  most  cases  to  make  use  of  the  fact  that  the  cumu¬ 
lative  of  a  Beta  distribution  corresponds  to  a  cumulative  of  a  binomial  distri¬ 
bution.  Thus,  for  example,  the  integral  from  0  to  1/2  of  the  Beta  distribution, 
with  parameters  a  and  b,  corresponds  to  the  sum  of  the  individual  terms  of  the 
binomial  distribution,  uith  oarameters  1/2  and  a  +  b,  and  the  sum  is  over  all 
terms  greater  than  or  equal  to  a  +  1.  The  binomial  distribution  has  been  extens¬ 
ively  tabled  in  a  number  of  places.  The  one  we  find  most  convenient  is  Tables  of 
the  Cumulative  Binomial  Probability  Distribution,' T' e  '  nnals  of  tie  Coi’-'utat  ion 
Laboratory,  Harvard  University,  Vol .  XXXV,  Cambridge,  Hass.:  Harvard  University 
Press,  1955. 

Thus,  the  expected  return  from  applying  the  instructional  sequence  optimally 
suited  to  uninformed  students  is  determined  by  first  finding  the  two  parameters 
of  the  Beta  distribution, which  represent  the  states  of  knowledge  of  the  population 
of  students,  and  determining  the  gain,  g,  and  the  cost,  c,  and  then  performing 
some  computations  to  find  the  mean,  p  ■  a/(a+b),  of  the  distribution  of  the  states 
of  knowledge  and  looking  up  two  values  of  the  cumulative  of  the  binomial  distri¬ 
bution. 


CONVENTIONAL  CHOICE  TESTING 

Consider  now  the  possibility  of  asking  each  student  a  question  before  we 
assign  him  one  of  the  three  instructional  sequences.  Suppose  further  that  this 
question  were  administered  by  one  of  the  conventional  choice  methods  which  allow 
the  student  only  two  responses,  where  his  choice  of  a  response  is  determined  by 
whether  his  state  of  knowledge,  p,  Is  greater  than  or  less  than  1/2.  Thus,  his 
response  to  the  question  is  a  two-valued  random  variable,  X,  where  X  ■  0  indicates  t 
that  the  student's  p  is  less  than  1/2  and  X  *  l  indicates  that  the  student's  p  is 
greater  than  1/2. 

Now,  we  can  compute  the  expected  return  given  that  the  student  responds  and 
that  one  of  the  instructional  sequences  is  applied.  Knowing  the  student's  response 
tells  us  whether  to  apply  the  left  half  or  the  right  half  of  the  Beta  distribution 
of  ability  levels  in  computing  the  expected  return.  If  we  consider  applying  lQ, 
the  return  Is  always  0.  Therefore,  the  expected  return  from  ij  is  always  0  whether 
or  not  the  student  responds  X  -  0  or  X  *  I .  Thus,  we  have 

(7a)  Exl1  -  0. 

For  applying  Iq,  if  a  student  responds  X  ■  0,  the  expected  return  is  the  Integral 
from  0  to  1/2  of  the  product  of  the  return  function  times  the  distribution  of 
ability  levels,  while  if  the  student  responds  X  ■  1,  It  is  the  Integral  from  1/2 
to  1  of  the  product  of  the  return  function  times  the  distribution  of  ability 
levels.  Thus, 


ft  [g(l-p)  -  c]f  (p|a,b)dp  If  X  -  0 
(7b)  E„I0.<  •  6 

IgO-p)  *  c]  f  g  (p  |  a  ,b)dp  If  X  -  1 
fg(l-p)Gb(a|i,  a+b)  -  cGb(a|i,a+b-l)  if  X  -  0 

\  g ( 1  -p) (b+ 1 |-,a+b)  -  cGb(b||,a+b-l )  If  X  -  1 

By  a  similar  process,  we  obtc  In  conditional,  expected  return  for  applying  1 ^ 

f  (gp~)f0(p|a,b)dp  if  X  -  0 

tg(l-p)  -  5]fg(p|a,b)dp 


(7c)  Exi,/2 


< 


If  X  -  1 


if  X  -  0 


gpGb(a+l  ||,a+b)  -  |Gfa (a | ^ ,a-*-b- 1 ) 

B  i 

'.g(1-p)Gb(b+1  Ij.a+b)  -  |G(b|J,a+b-l)  if  X  =  1 
And  finally,  the  probability  of  a  student  responding,  X,  is 

i  r  Mp|a,b)dp  for  X  *  0  »  G,  (a|:,a+b-l ) 

(7d)  P(X)  -  <J  J  8  b  2 

[  f±  fg (p | a ,b)dp  for  X  *  1  -  Gb(b|j;,a+b-1 ) 

ADMISSIBLE  CHOICE  TESTING 

Remember  from  Section  A  above  that  the  use  of  an  admissible  choice  procedure 
yields,  at  least,  three  different  responses,  Y  *>  0,1,2.  If  a  student  answers  Y  *  0, 
he  is  misinformed  about  the  topic  under  consideration.  If  the  student  answers 
Y  *  I,  he  is  neither  very  misinformed  nor  vory  well-informed  about  the  topic  under 
consideration.  If  the  student  answers  Y  ■  2,  he  Is  fairly  well-informed  about 
the  topic  under  consideration.  By  varying  the  size  of  the  penalty  for  a  wrong 
response,  we  can  vary  the  critical  values  of  p,  where  the  student  will  make  one  or 
another  response.  To  be  explicit,  we  will  consider  the  case  in  which  the  student 
wi 1 1  respond  Y  =  0,  if  p  is  less  than  1/4;  he  wi 1 1  resoond  Y  ■  1,  if  o  is  between 
1/4  and  3/4;  and  he  will  respond  Y  ■  2,  if  p  is  greater  than  3/4.  Remember  that 
the  middle  category  can  be  broadened  or  i arrowed  and  would  have  some  effect  on  the 
performance  of  admissible  choice  as  a  guide  to  instruction,  depending  upon  the 
exact  values  of  g  and  c.  However,  1/4  and  3/4  are  not  too  unreasonable  to  use  in 
general  with  admissible  choice  testing  and,  in  reality,  we  are  probably  not  free 
to  change  the  penalty  (and,  thus,  the  middle  range)  from  time  to  time  and  expect 
the  students  to  adjust  immediately  to  the  change. 

Consider  now  the  expected  return  from  first  asking  a  question  about  the  topic 
under  consideration  by  using  an  admissible  choice  procedure  and  then  apnlyine  one 
of  the  three  instructional  sequences  depending  upon  the  answer  that  the  student 
gives.  As  before,  the  expected  return  from  applying  1^  is  0 

(8a)  Evl j  -  0 


* 


I 


We  obtain  the  expected  return  by  applying  lg  In  a  similar  fashion  to  that  given 
for  conventional  choice  testing,  but  for  three  different  responses 

( ft  l9(  1-p)  -  o]f  (p|a,b)dp  if  Y  «*  0 


(8b)  Eyl0  ■  \  ff  Cg ( 1 -p)  -  c]fft (p|a,b)df 


[gO-p)  “  c]fg(p|a,b)dp 


if  Y  -  1 

If  Y  -  2 


g(l-p)Gb(a |^,a+b)  -  cGb(a|£,a+b-l)  if  Y  «  0 


g(l-p)  "  c  fY-o'o  ‘  EY-2 1  ( 


If  Y  -  1 


U(l-p)Gb(b+l|J,a+b)  -  cGb(b|J,a+b-l)  if  Y  -  2 


and  the  expected  return  from  applying  i ^ 

{  if  [gp  -  |] fg (p| a  ,b)dp 


(Oc)  Eyi,/2 


if  Y  -  0 


if  [f;P  ■  \]  fg (p |a ,b)dp  +  [p(l-p)  -  |]fp(o|a,b)dp  if  Y 

JjL  [g(l-p)  -  |l  "Pg  (p  |  a  ,b)  dp  if  Y  «  2 


’gpGb(a+l|J,a+b)  -  fnb(a|J,a+b-l)  if  Y  -  0 

gpGb(a+l|i,a+b)  -  §Gb<a|i,a+b-l)  -  ^]/2 
+  g(l-p)Gb(b-H|i,a+b)  -  |r,b(b|j,a+b-l)  -  FY<l2»  l/2  if  Y 


Vg(l-p)Gb(b-H|j,a+b)  -  |Gb(b|j  a+b-1)  If  Y  -  2 


Likewise,  the  probability  of  students  responding  Y  -  0,1,2  is 


^  fg(p|a,b)dp 

for  Y  -  0 

Gb(a||,a+b-l) 

(  fg(p|a,b)dp 

i  * 

for  Y  -  1 

Gb(a||,a+b-l)  +  Gb(b|j,a+b-l) 

l  fg(p|a,b)dp 

for  Y  »  2 

Gb(b|{,a+b-i) 

MATCH  I  MG  INSTRUCT  I  Of)  TO  THE  AVERAGE  STUDENT 

Consider  now  the  last  Instructional  strategy,  that  of  matching  instruction  to 
the  average  student.  To  he  more  explicit,  the  idea  is  to  determine  the  average 
state  of  knowledge  of  the  students  and  then  to  select  and  apply  that  Instructional 
sequence  precisely  tailored  to  this  mean  value.  Though  there  Is  some  logic  to  this 
procedure,  It  has  been  widely  criticized,  probably,  most  often  because  people 
object  to  the  idea  of  using  one  Instructional  sequence  for  pupils  of  varying  states 
of  knowledge.  It  has  been  criticized  somewhat  less  often  because  people  object  to 
matching  Instruction  to  the  mean  rather  than  to  some  other  value.  In  this  report, 
we  will  just  consider  matching  instruction  to  the  mean  and  in  a  later  renort  we 
may  consider  matching  Instruction  to  other  aspects  of  the  distribution  of  states 
of  knowledge. 

To  consider  this  case,  we  have  to  specify  our  cost,  oaln,  and  return  functions 
more  renerally,  since  any  orecisely  tailored  instructional  sequence,  1^,  may  be 
chosen  and  misapplied  to  students  with  various  states  of  knowledge. 

it  Is  only  logical  to  use  the  same  cost  function  that  we  used  before  for  a 
precisely  tailored  instructional  sequence,  since  it  depends  uoon  the  Instructional 
sequence  and  not  upon  the  state  of  knowledge  of  the  student.  Thus, 


(9)  Cost  ■  c(l-p) 


'•/e  must,  however,  more  completely  specify  the  return  function.  To  see  this,  suppose 
we  apply  I For  students  whose  states  of  knowledge,  p,  are  nreater  than  3A. 
the  return  function  will  be  similar  to  those  specified  before  and,  due  to  the  limit 
on  the  maximum  state  of  knowiedqe,  the  return  will  decline  as  p  approaches  1.  For 
students  whose  states  of  knowledge,  p,  are  less  than  3A,  the  return  will  decline 
as  p  Increases.  Assuming  that  Instruction  is  progressively  less  effective  in 
increasing  the  students*  states  of  knowledge  as  the  instruction  is  tailored  for 


students  who  are  at  progressively  higher  levels  of  knowledge,  then  we  can  approxi¬ 
mate  this  behavior  by  a  linear  function  which  declines  at  a  certain  rate.  The 
decline  progresses  down  to  where  the  gain  is  zero  and  then  remains  at  zero  for  all 
smaller  values  of  p.  Thus,  we  can  derive  a  set  of  return  functions  which  Includes 
those  used  earl ler 


-c(l-p)  for  p  <  2p  -  1 

(10)  |-(p)  -  gp  -  g(2p-l)  -  c(1-p)  for  2p  -  1  <  p  <  p 

O  —  — 

g(l-p)  -  c(l-p)  for  p  >  p 

Three  different  return  functions  are  Illustrated  In  Figure  h  along  with  the  return 
function  for  precisely  tailored  Instruction. 

Using  this  general  Equation  10  for  the  return  function,  we  can  compute  the 
expected  return  from  using  an  Instructional  sequence  precisely  tailored  to  the 
mean  state  of  knowledge  of  the  students.  Thus, 

(11)  E  l-(p)  -/  l-(p)f0(p|a,b)dp 

=  ]2P-1 [-c(l-p)]f0(p|a,b)dp 

+  /P.  lgp-g(2p-0  “  c(l-p)]f.(p|a,b)dp 
2p-l 

+  /_  [q(l-p)  -  c(l-p)]fg(p|a,b)dp 

P 

"  gp(Fg(p|a+l  ,b)  -  Fg(2p-1 |a+l ,b)  } 

-  g(2p-l) (F0(p|a,b)  -  Fg(2p-1 |a,b) } 

+  g(l-p)G0(p|a,b+1)  -  c(l-p) 
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Figure  *».  Illustrative  return  functions  for  matching  Instruction 
£o  average  state  of  knowledge. 


SOMF.  COMPUTED  VALUES  BASED  ON  SELECTED  PARAMETERS 


In  order  to  get  some  numbers  from  these  equations,  we  have  to  qlve  numerical 
values  to  four  parameters.  Two  of  them,  a  and  b,  specify  the  distribution  of 
states  of  knowledge  of  the  students  under  instruction.  The  other  two,  g  and  c, 
represent  gain  and  cost,  respectively,  of  the  instruction.  We  will  fix  g  *  k 
and  c  ■  2  for  all  of  the  following  computations.  The  results  are  a  little  more 
general  than  they  seem,  since  they  also  apply  proportionally  to  all  cases  where 
g  *  2c. 

We  use  seven  different  distributions  of  states  of  knowledge.  Three  of  them 
are  symmetric  about  p  ■>  1/2,  but  with  different  degrees  of  spread  about  this  value. 
Two  are  skewed  with  many  of  the  students  being  misinformed,  while  the  other  two 
distributions  are  skewed  in  the  other  direction,  with  many  of  the  students  being 
well-informed.  The  following  pages  give  the  parameters  and  the  computed  vaiues 
along  with  comments  on  the  results. 


Parameters:  a  ■  b  ■  1,  g  ■  4,  c  *  2,  p=  1/2 


Case 

1: 

E  1  - 

P 

1 

Case 

II: 

E  'o 

=  0 

E  1 1/2 

-  0 

E  >| 

-  0 

Case 

III: 

(X  *  0) 

(X  - 

1) 

Ex'o 

*  *5 

-.5 

Ex 1 1  n 

-  0 

£ 

Ex'i 

**  0 

0 

P(X) 

■  .5 

.5 

Case 

IV: 

r  -  o) 

(Y  *  1 

) 

CM 

a 

> 

Fv'0 

-  .375 

0 

-.375 

Ey'|/2 

—  .125 

.25 

LTV 

CM 

1 

Eyl, 

-  0 

0 

0 

P(Y) 

-  .25 

.50 

.25 

Case 

V: 

E  1- 

*  0 

P 


Eyl 


* 


.5 


* 


Eyl 


.625 


Comments:  Here  the  distribution  of  states  of  knowledge  corresponds  to  the  rectan¬ 
gular  distribution  shown  in  Figure  2.  Effectively,  the  teacher  has  no  information 
whatsoever  concerning  the  student's  state  of  knowledge.  Notice  that  precisely 
tailored  instruction  yields  a  (^turn  of  some  size,  while  tailoring  instruction 
with  no  testing  either  by  choosing  one  of  the  three  sequences  or  by  matching 
instruction  to  the  average  student  yields  a  return  of  0.  The  use  of  choice 
testing  to  guide  instruction  yields  a  return  of  half  of  that  possible  by  the 
use  of  admissible  probability  measurement  procedures  while  the  use  of  admissible 
choice  procedures  yields  a  return  somewhat  higher  than  that  of  conventional 
choice  testing.  In  summary,  in  this  case  the  teacher  has  no  information  whatso¬ 
ever  about  the  students,  so  not  surprisingly,  testing  nays  off. 


Parameters:  a  =  b 

■  3,  g  -  **, 

c  -  2,  p  -  1/2. 

Case  1 :  El 

P 

-  1 

Case  11:  E  lQ 

«  0 

E  1 1/2 

-  .375 

E  'l 

-  0 

Case  III: 

X  -  0 

X  »  1 

A 

Ex'  " 

O 

X 

UJ 

-  .3125 

-.3125 

E  X ' 1 /2 

-  .1875 

.1875 

m 

X 

-  0 

0 

P(x) 

“  .5 

.5 

Case  IV: 

y  «=  0 

Y  -  1 

Y  -  2 

ey'o 

w  .13182 

0 

-.13182 

EY ' 1/2 

-  -.C2832 

.1»316A 

-.02832 

ey'i 

-  0 

0 

0 

p  (y) 

*  .10352 

.79296 

.10352 

Case  V:  E  1. 

P 

-  -375 

EyI*  *  .563^6 


Comments:  The  distribution  of  the  students'  states  of  knowledge,  in  this  case, 
may  be  seen  in  Figure  2.  It  is  symmetric  about  p  ■  1/2  with  a  maximum  at  this 
value.  The  students'  states  of  knowledge  fall  over  a  broad  range,  but  relatively 
few  are  very  well-informed  or  very  misinformed.  Notice  that,  in  this  case, 
precisely  tailored  instruction  again  yields  a  return  of  some  size.  In  tailoring 
instruction  without  testing,  1^  is  the  best  of  the  three  and  yields  an  expected 
return  equal  to  the  expected  return  of  matching  instruction  to  the  average  student. 
As  before,  the  use  of  choice  testing  to  guide  instruction  yields  hair  the  expected 
return  from  using  admissible  probability  measurement.  The  use  of  an  admissible 
choice  procedure  is  somewhat  better,  but  the  gain  is  less  than  was  found  before. 


Parameters:  a  •*  b  =  20,  g  *>  4,  c  »  2,  p  =  1/2. 


Case  1 : 

El  -  1 

P 

Case  1  1 : 

E  'o  "  0 

E  1  1  /2  “  .7l<92l< 

El.  -  0 

Case  III: 

X  =  0 

X  -  1 

Exl*  -  .74924 

Exl0  -  .12538 

-.12538 

Ex'  1/2  -  •37'162 

.37462 

Ex'i  =  0 

0 

P(x)  «=  .5 

.5 

Case  IV: 

Y  =  0 

Y  =  1 

Y  =  2  Eyl*  -  .74970 

Ey  1 0  -  .0001*0 

0 

-.00040 

Ey‘ 1/2  =  -.00003 

.74930 

-.00003 

ey'i  -  0 

0 

0 

P(Y)  -  .00037 

.99926 

.00037 

Case  V: 

E  1-  =  .74924 

P 

Comments:  This  distribution  of  the  students'  states  of  knowledge  can  also  be 
found  in  Figure  2,  It  is  the  very  narrow  distribution  symmetric  about  p  “  1/2, 
the  great  majority  of  students  being  in  the  region  of  being  uninformed,  but 
essentially  no  one  is  well-informed  or  misinformed.  It  represents  a  very 
homogeneous  class  which  may  have  arisen  either  through  a  previously  applied 
placement  process  or  because  of  the  nature  of  the  subject  matter  under  study, 
such  as  Greek,  where  relatively  few  students  would  possess  prior  knowledge  of  the 
subject.  Here,  the  expected  return  from  precisely  tailored  instruction  is  of  some 
size  which,  again,  is  best  of  any  of  the  procedures.  Notice,  however,  that  all 
the  remaining  procedures  do  about  equally  well.  One  can  do  as  well  without 
testing  as  by  using  a  conventional  choice  test  and  the  gain  from  admissible  choice 
testing  is  really  trivial.  This  is  a  reflection  of  the  vast  amount  of  information 
already  known  about  the  students'  states  of  knowledge.  In  summary,  as  the  class 
becomes  more  homogeneous,  with  relatively  fewer  well-informed  and  misinformed 
students,  the  value  of  precisely  tailored  instruction  remains  best  and  constant 

with  the  return  from  the  other  procedures  increasing  until,  finally,  the  gain  from 
choice  testing  is  wiped  out,  but  with  considerable  advantage  still  remaining  to 
the  use  of  admissible  probability  measurement. 


Parameters 

:  a  **  1  ,  b 

a  3,  g  »  4,  c 

=  2,  p  ■  1/4. 

Case  1 : 

E  1 

P 

1.5 

Case  1 1 : 

E  'o  " 

\_ 

E  ' 1 /2  " 

-.25 

Case  111: 

E  >1  ‘ 

0 

X  “  0 

X  =  1  Exl*  -  1.125 

Vo  “ 

1 .0625 

-.0625 

EX*  1/2  ** 

-.1875 

.0625 

Ex'i  - 

0 

0 

p(x) 

.875 

.125 

Case  IV: 

Y  **  0 

Y  -  1  Y  =  2  Eyl*  *  1.08982 

ey’o 

.89451 

.12502  -.03126 

Vl/2  " 

-.31641 

.19531  -.00390 

Vi  - 

0 

0  0 

P(Y) 

.57813 

.40624  .01563 

Case  V: 

E  1- 
P 

.86721 

Comments:  This  distribution  of  students'  states  of  knowledge  is  also  shown  in 
Figure  2.  It  is  the  moderately  positively  skewed  distribution  with  relatively 
few  of  the  students  being  well-informed.  Here,  the  return  from  precisely  tailored 
instruction  is  even  greater  than  before.  This  is  primarily  a  reflection  of  the 
characteristic  of  the  return  functions  that  the  yield  is  greater  from  training 
a  misinformed  student.  In  the  case  of  selecting  one  of  the  three  instructional 
sequences  without  testing,  it  is  best  to  treat  all  of  the  students  as  being 
misinformed  and  the  return  is  2/3  of  that  obtainable  from  precisely  tailored 
instruction.  Notice  that  matching  instruction  to  the  average  student,  i.e., 
applying  1.  yields  almost  as  great  a  return  as  does  using  lQ.  The  use  of 

choice  testing  is  slightly  better  than  applying  1^.  The  use  or  admissible  choice 
testing,  while  very  slightly  better  than  applying0^,  is  not  quite  as  good  as 
using  conventional  choice  testing.  This  is  undoubtedly  due  to  the  choice  of  the 
penalty  score  for  the  admissible  choice  testing  procedure  used.  Admissible  choice 

testing  applies  lft  whenever  a  student  picks  the  wrong  answer,  which  is  about  58% 
of  the  time  and  applies  I j  whenever  the  student  skips  the  question,  which  is 
about  41%  of  the  time,  whereas  conventional  choice  testing  applies  1^  whenever  the 
student  gets  the  wrong  answer,  which  is  about  88%  of  the  time  and  applying  I j 
whenever  the  student  gets  the  right  answer,  which  is  about  12%  of  the  time. 

In  summary,  when  many  of  the  students  are  not  well-informed,  choice  testing  is  of 
little  value.  The  use  of  admissible  probability  measurement  with  precisely 
tailored  instructional  sequences  still  retains  a  considerable  superiority. 
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Comments:  This  distribution  of  students'  states  of  knowledge  is  also  shown  in 
Figure  2.  It  is  the  highly  positively  skewed  distribution  with  almost  all  the 
students  being  more  or  less  misinformed.  Such  a  situation  might  be  faced  by 
a  teacher  given  the  task  of  political  indoctrination  or  of  teaching  a  group  of 
students  selected  by  a  previously  applied  placement  process.  Here  the  return 
from  precisely  tailored  instruction  is  somewhat  greater  than  previously, 
primarily  reflecting  the  greater  number  of  students  who  are  misinformed.  Notice, 
however,  that  all  the  other  instructional  strategies  yield  almost  the  same  return. 
This  is  so  because  they  recommend  essentially  the  same  instructional  sequence. 

They  all  recommend  the  application  of  Iq  except  in  the  case  of  matching  instruction 
to  the  average  student.  In  this  case,  the  recommendation,  I  is  not  too 
different  from  lQ.  So,  in  this  case,  testing  doesn't  help,  whether  it  is  a  choice 
procedure  or  admissible  probability  measurement.  This  is  primarily  because  the 
amount  of  information  already  available  to  the  teacher  is  just  too  great  and 
additional  information  will  not  help. 
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Comments: 
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Figure  2, 
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the  reverse  of  the 

distribution  for  a  -  1,  b  -  3 

In  other  words,  It  is  negatively  skewed  with  relatively  few  of  the  students  being 
misinformed.  Here,  the  return  from  precisely  tailored  instruction  is  less  than 
before,  essentially  reflecting  the  fact  that  the  students  need  less  instruction 
than  before.  In  the  case  of  choice  between  the  three  instructional  sequences 
without  testing,  the  best  choice  Is  no  Instruction,  but  the  yield  is  zero.  Notice 
also  that  considerable  loss  results  from  applying  either  I ^  and,  particularly,  1^. 
Matching  instruction  to  the  average  student  does  almost  as  badly  with  a  very  small 
yield.  Choice  testing  is  even  worse  than  matching  to  the  average  student,  while 
admissible  choice  testing  does  pretty  well  by  recommending  1^,  whenever  the  student 
chooses  the  wrong  answer;  l|/2*  whenever  the  student  skips  the  question;  and  lj, 
whenever  the  student  answers  correctly. 
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Comments:  This  distribution  is  not  shown  in  Figure  2,  but  it  is  the  reverse  of  the 
distribution  for  a  *  1,  b  =  19.  In  other  words,  It  is  highly  negatively  skewed 
with  almost  all  of  the  students  being  fairly  well-informed.  Here,  the  return  from 
precisely  tailored  instruction  is  not  very  large  compared  to  the  values  previously 
obtained.  Application  of  one  of  the  three  instructional  sequences  without  testing 
yields  a  return  of  zero  from  applying  lj,  that  is,  no  instruction.  The  misappli¬ 
cation  of  Iq  or  yields  very  large  losses,  however.  Choice  testing  is  of  no 

help  with  a  truly  trivial  advantage  for  admissible  choice  testing  over  conventional 
choice  testing.  Matching  instruction  to  the  average  student  actually  yields  a 
negative  return,  but  of  very  small  value.  In  summary,  none  of  the  returns  here  are 
very  large  compared  to  those  obtained  previously.  However,  there  are  undoubtedly 
situations  in  which  It  is  very  important  to  have  all  the  students  very  well-trained 
(e.  g.  maintaining  proficiency  for  critical  jobs)  so  that  the  return  would  become 
quite  important.  In  such  a  case,  it  is  evident  that  this  type  of  training  cannot 
be  achieved  efficiently  without  the  use  of  the  precisely  tailored  instruction  based 
on  admissible  probability  measurement,  at  least  not  by  the  use  of  any  of  the 
procedures  considered  here. 


CONCLUSIONS 


Precisely  tailored  instruction  has  evidenced  a  marked  superiority  to  the 
other  methods  in  all  instances  considered  so  far.  Now,  precisely  tailored  instruc¬ 
tion  is  not  characteristic  of  most  formal,  institutional  procedures  for  instruction, 
it  has  certainly  never  been  used  in  conjunction  with  admissible  probability 
measurement.  Precisely  tailored  instruction,  in  the  spirit  in  which  it  is  used 
here,  is  probably  best  approximated  by  very  qood  tutorial  instruction  in  the  case 
of  external  instruction  and  by  a  graduate  student  or  by  a  self-taught  expert  in  the 
case  of  internally  guided  instruction.  Most  talk  about  individualized  instruction 
is  in  the  context  of  institutionalized  instructional  programs  and,  in  this  context, 
individualized  instruction  has  been  in  the  past,  at  most,  something  approximating 
the  use  of  choice  testing  to  guide  the  application  of  special  instructional 
sequences.  This  kind  of  instruction  has  been  approximated  in  some  schools  and  by 
programmed  textbooks  and  computer-based  materials  which  branch  on  the  basis  of 
the  students'  choices.  Most  programmed  instruction  and  computer-assisted  instruc¬ 
tion  approximates  the  matching  of  instruction  to  the  average  student,  since 
presumably,  the  instructional  material  is  evolved  through  successive  tryouts  to 
do  a  fairly  good  job  with  most  students.  This  also  applies  to  some  extent  to  the 
development  of  textbooks  and  other  instructional  material.  However,  the  choice 
between  textbooks  and  other  instructional  material,  as  faced  in  most  schools,  is 
approximated  by  a  priori  choice,  where  no  testing  is  done  and  previousiy  available 
information  about  the  students  is  used  to  select  one  of  three  instructional 
sequences . 

In  general,  we  find  that  individualized  instruction  is  at  least  as  nood  as 
these  other  procedures  already  in  use.  In  many  cases,  however,  it  is  no  better 
and,  in  other  cases,  it  is  probably  not  enough  better  to  offset  the  additional 
cost  of  testing.  Matching  instruction  to  the  average  student  is,  frankly,  better 
than  v/e  thought  it  would  be.  it  is  as  good  as  individualized  instruction  in  every 
case,  but  two.  The  most  dramatic  instance  is  where  the  teacher  has  essentially 
no  information  about  the  students'  states  of  kno  ledge.  Th  other,  less  dramatic, 
instance  is  where  relatively  few  of  the  students  are  well-informed,  but  there  is 
still  a  moderate  spread  in  the  students'  states  of  knowledge  with  the  majority  of 
students  tending  to  be  somewhat  misinformed.  This  latter  instance  seems  somewhat 
less  likely  to  occur  in  actual  practice  than  the  first  instance,  which  might  be 


approximated  by  those  situations  in  which  the  teacher  is  teaching  a  very  large 
class  of  students  or  in  which  the  teacher  is  beginning  a  course  of  instruction  with 
new  students  about  which  he  has  no  prior  information.  Thus,  it  would  appear  that 
individualized  instruction,  as  conceived  by  most  educators,  would  be  of  greatest 
benefit  in  these  situations,  in  these  and  many  other  situations,  however,  the 
benefit  from  precisely  tailored  instruction  would  be  much  oreater.  One  of  the 
hurdles  to  achieving  precisely  tailored  instruction  in  a  formal,  instructional 
context  has  been  surmounted  by  the  develooment  of  admissible  probability  measure¬ 
ment  procedures.  Another  major  hurdle  probably  lies  in  mal'inn  the  application  of 
the  precisely  tailored  instructional  sequences  economically  and  organizationally 
feasible.  Computer  based- instruct  ion,  tutorial  testing,  and  independent  self-study 
probably  hold  some  promise  here.  ’?«  plan  to  investigate  their  potential  in  a 
future  report. 
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